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Abstract: 

A feedback neural network approach to communication routing problems is developed with emphasis 
on Multiple Shortest Path problems, with several requests for transmissions between distinct start- 
and endnodes. The basic ingredients are a set of Potts neurons for each request, with interactions 
designed to minimize path lengths and to prevent overloading of network arcs. The topological 
nature of the problem is conveniently handled using a propagator matrix approach. Although the 
constraints are global, the algorithmic steps are based entirely on local information, facilitating 
distributed implementations. In the polynomially solvable single-request case the approach reduces 
to a fuzzy version of the Bellman-Ford algorithm. The approach is evaluated for synthetic prob- 
lems of varying sizes and load levels, by comparing with exact solutions from a branch- and-bound 
method. With very few exceptions, the Potts approach gives legal solutions of very high quality. 
The computational demand scales merely as the product of the numbers of requests, nodes, and 
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Introduction 



Communication routing resource allocation problems are becoming increasingly relevant given the 
upsurge in demand of internet and other telecommunication services. One such problem amounts 
to assigning arcs (links) in a connected network to requests from start- to endnodes, given capacity 
constraints on the links, such that a total additive cost (path-length) is minimized. For a review 
of notation and existing routing techniques, see e.g. ref. A relatively simple routing problem, 
with only one request at a time, is the Shortest Path Problem (SPP), which can be solved exactly 
in polynomial time using e.g. the Bellman Ford (BF) algorithm m. The Multiple Shortest Path 
Problem (MSPP), where links are to be allocated simultaneously to several requests, is more 
difficult. There exists, to our knowledge, no method that yields exact solutions to this problem in 
polynomial time. 

In this paper we address the MSPP using feedback Potts neural networks, which have proven to be 
powerful in other resource allocation problems, with [0 or without Q a non-trivial topology. For 
each request we assign a Potts network, with units encoding which links to be used by that request. 
Appropriate energy terms are constructed in terms of the Potts neurons to minimize total path 
lengths and to ensure that capacity constraints are not violated. Mean field (MF) equations are 
iterated using annealing to minimize the total energy. In contrast to earlier usage of Potts encoding 
and MF annealing [Q, ||, |^| where global objective functions are minimized, here each node minimizes 
its own local energy. 

For the case of a single request, the Potts MF approach reduces in the zero temperature limit to 
the BF algorithm; hence our approach contains this standard algorithm as a special case. 

For each request the Potts MF network jl| defines a "fuzzy" spanning tre^ rooted at the endnode. 
In order to project out the (fuzzy) path from the startnode in this subgraph, and to keep track of the 
paths in general, we utilize a propagator matrix formalism following ref. Q. The computation of 
the propagator requires matrix inversion; fortunately this can be done using an iterative procedure 
with low computational cost. 

As in a previously considered airline crew scheduling problem , proper preprocessing is employed 
to identify independent subproblems, in order to reduce the problem complexity. 

Despite the existence of global constraints, the implementation of the approach is truly local - when 
updating the MF equations for a particular node, only information residing at neighbouring nodes 
is needed. 

The approach is gauged by an exact branch-and-bound (BB) algorithm on a set of synthetic but 
challenging test problems, showing an excellent performance of the Potts MF approach, with a CPU 
consumption per request scaling merely like NNl, where N is the number of nodes and Nl the 
number of links in the network. The method is also very robust with respect to parameters. Due to 
the excessive demand for CPU resources by the reference BB method, our comparisons are limited 
to fairly low problem sizes. However, there are no indications that the Potts method be less efficient 
for larger problems. 



5 A subgraph connecting all nodes without loops. 
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Figure 1: Example of a solution to a 3-request problem. Dotted lines represent unused links, and 
full lines links used by the requests. 

The Multiple Shortest Path Problem 

A Multiple Shortest Path Problem (MSPP) is defined by specifying the following: 

• A connected network of N nodes and Nl links (arcs). 

• For each arc ij, a cost (arc-length) and a capacity CV,-. 

• A set of Nr transmission requests r, each defined by a startnode a r and an endnode b r . 

The task is then to assign to each request a connected loop-free path of arcs from the startnode to 
the endnode. This is to be done such that the total length of those paths is minimized, without the 
load on any arc exceeding its capacity, with the load defined as the number of requests sharing it. 
A 3-request problem example is given in fig. [j]. 

All problems of this kind are not solvable; a reliable algorithm should be able to recognize and signal 
a failure, to enable proper measures to be taken. 



The Bellman-Ford Algorithm in the Mean Field Language 

Prior to dealing with the MSPP, we revisit the simpler SPP and demonstrate how the BF algorithm 
can be recast in a Potts MF language. This formulation will then be the starting point for designing 
a Potts MF approach to MSPP. 

In the SPP there is only a single request, from a to 6, and the capacity constraints are irrelevant. 
The task simply is to find the shortest path from a to b. In the BF algorithm Q this is achieved by 
minimizing the path-lengths Di from every node i to 6, by iterating 

Di —> mm(dij +Dj), i^b. (1) 

3 

and keeping track of the involved links ij. Db is fixed to zero by definition. The resulting solution 
defines a spanning tree rooted at b. In particular, D a is determined, and the minimal path from a 
to b is easily extracted from the spanning tree. 
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If no link exists from node i to j, dij could formally be denned to be infinite; in practice it is more 
convenient to restrict j in eq. (|l|) to the actual neighbours of i, reachable via an arc from i. 

Eq. (|l|) can be rewritten as 

Di = Vij Eij = ^ Vij (dij + Dj ) (2) 

3 3 

in terms of a Potts spin v, for every node i ^ b, with components wy taking the value 1 for the j with 
the smallest local energy E^, and for the others (winner-takes-all). Note the distinct philosophy 
here: each node i minimizes its own local energy Di — minj E^, rather than all nodes striving to 
minimize some global objective function. 

A mean field (MF) version of eq. (||) is obtained by using for Vj its thermal average in the MF 
approximation, defined by 

e -Eif/T 

where j and k are neighbours of i, and T is an artificial temperature. Note that each Potts MF 
neuron v, obeys the normalization condition 

3 

allowing for a probabilistic interpretation of the components. 



At a non-zero temperature, iteration of eqs. (|2],|3|) can be viewed as a fuzzy implementation of the 
BF algorithm, while in the T — > limit the neurons are forced on-shell, i.e. vtj — ► 1 (for the 
minimizing j) or (for the rest), and proper BF is recovered. 

Given this obvious neural recast of the BF algorithm in terms of Potts neurons, it is somewhat 
surprising that non-exact neural approaches based on Ising spins have been advocated in the liter- 
ature [Q. 

In addition, the MF Potts algorithm considered here exhibits a close electrostatic analogy in terms 
of Kirchhoff's laws on a graph M. 



The Potts Mean Field Approach to MSPP 

The Potts MF formulation of the Bellman- Ford algorithm for SPP (eqs. (0,^|)) is a suitable starting 
point for approaching MSPP. 

We will stick with the philosophy inherited from BF of focusing on independent local energies, in 
contrast to what has become standard when using feedback neural networks for resource allocation 
problems. This represents a novel strategy. 

Thus, we introduce a separate Potts system, {u^}, for each request r, with basic local energies -Ey- 
as before representing distances to the endnode b r . In addition, we will need energy terms E\° for 
the load-constraint, to be discussed later; this introduces an interaction between the Potts systems. 



3 



Figure 2: Expansion of the propagator PT. in terms of wjy ( — ) and intermediate nodes (•). 



This formulation introduces the possibility of undesired loop formation, since forming a loop might 
induce less energy penalty than violating a load constraint. As will be discussed below such loops 
can be suppressed by a suitable penalty term, i?]° op , and by adding a possibility for each proper 
node to connect, via an artificial escape link, to an artificial escape node, for each request connecting 
directly to the endnode. This enables a "give-up" state for an unresolvable situation, signaled by 
some path containing the escape node. The cost for "giving up" must be larger than that for any 
legal path. Therefore the length of each escape link is set to the sum of the lengths of the proper 
links, while the corresponding capacity is chosen large enough to be able to host all the requests. 

In order to terminate the path for a request r, its endnode must be a sink for the corresponding 
Potts system. Consequently, there will be no Potts neuron v£ associated with it. 

In order to construct appropriate penalty terms, a propagator matrix will be used. This technique 
has proved to be a powerful tool in neural optimization for problems with a non-trivial topology 
0. In particular, it will be crucial for extracting properties of the fuzzy paths defined by the MF 
approach at finite T. 



Path Extraction and the Propagator 

The normalization condition (eq. (Q)) ensures that for each request there is precisely one continu- 
ation for each node, although for T ^ it is fuzzily distributed over the available neighbours. On 
shell, the path from start- to endnode is trivial to extract - one follows the v\a = 1 path starting 
from the startnode. However, for T^Oa more refined path extraction mechanism is needed. This 
is provided by a propagator matrix Q P r for each request r, defined by: 



P r = 



(l-v r ) 1 =S ij + vl j + Y i v r ik vL + Y / vl k v r kl v'[ j + ... (5) 



For a graphical representation, see fig. ||. On shell, it is easy to see that PL can be interpreted 
as the number of paths from i to j defined by the Potts neurons associated with r; similarly, the 
elements of the matrix square (P r )fj are related to the number of arcs used in those paths[j. 

Off shell, these interpretations are still valid in a probabilistic sense. A probabilistic measure of how 
much node i participates in the path a r — > b r is given by PT ^PT, = P^ fJ the simplification being 
due to the identity P[ b = 1, following from eq. (^). This provides us with a natural path extractor; 
in particular, we have on shell (in the absence of loops) 

p r _ ( 1, if node i appears in the path a r — > b r 
a ' z 1 0, otherwise 



More precisely, the number of arcs is given by P 2 — P. 



Load and Loop Control 



Armed with the propagator formalism, we proceed to set up the penalty terms, to be added to the 
energies Eij corresponding to eq. (||) . 

In order to efficiently avoid complications from self-interactions in the local energies, independently 
of T, we follow ref. ||, and define the penalty terms based on analyzing the result of setting one 
component vjj of the neuron v[ at a time to one, with the other components set to zero, as compared 
to a reference state with all components set to zero. 

The total load on a link ij is the sum of contributions from the different requests, 

Y. L 'r ( 7 ) 

r 

The contribution from request r can be expressed as 



• 7^<-:, (8) 



where corrects for possible improper normalization due to soft loop-contributions. 

The load constraints, Lij < Cij, define a set of inequality constraints. In the realm of feedback 
neural networks, such constraints have been successfully handled by means of step-functions Q. 
For a given request r, the available capacity of the link ij is given by 

X^Cij-Lij + L^, (9) 

in terms of which an overloading penalty can be defined as 

E\™ d = (i-x)e{i-x) + {x)<d(-x) (io) 



where 0() is the Heaviside step-function. Eq. (10) expresses the additional overloading of the link, 
if it were to be used by r. 

The amount of loops introduced by connecting i — * j can be expressed as 

pr 

Y=-^ (11) 

ii 

A suitable loop suppression term is then given by 

^■ op - r^y- ( 12 ) 



The generalization of the local energy in eq. (g) to the multiple request case now reads, for a 
particular request r, 

E^ ^d l , J +D r J +aE l ° aA + 1 E x ° ov (13) 

with the added terms based on eqs. ( |To| ? p~2[ ) . The resulting algorithm allows for a wide range of 
choices of the coefficients a and 7 without severely changing the performance. 
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Updating Equations 



All neurons are repeatedly updated, with a slow annealing in T. For each request r and each node 
i, the corresponding neuron vj is updated according to 

e-Etj/T 

(14) 



with given by eq. (|l3|). The corresponding length D\ from node i to the endnode is then 
updated, in the BF spirit, as 

IK -Y. >■:,!■:„. (15) 

3 

The corresponding update of the propagator could in principle be done using an exact incremental 
matrix inversion scheme like Sherman-Morrison ||. We prefer, though, to let local changes propagate 
through the network, in analogy to the update of D\. Thus, only the i'th row of P r is updated: 

PL^S m + J2^PJ m , for all m (16) 

3 

This gives a convergence towards the exact inverse, which turns out to be good enough. The 
advantage of this method is twofold: it is faster, and all information needed is local to the relevant 
node i and its neighbours j (assuming each node to keep track of its own row of P' ). 

Details of the algorithmic steps and the initialization can be found in Appendix A. 



Test Problems and Explorations 



In order to test the Potts MF method we have generated a set of synthetic problems. In doing so 
we face limitations given by the ability of the exact reference method (BB) to provide solutions, 
due to limited computing power. For this reason our comparisons are restricted to relatively small 
systems. 

With these limitations in mind, we have tried to choose the test problems as difficult as possible. 
The most important parameters governing the difficulty of a problem, apart from network size and 
connectivity, are the number of requests and the average link capacity. In cases where all the links 
are able to host all the requests, Cij > Nr, the problem is separable, and can be solved by running 
an independent BF algorithm for each request. Therefore we choose to work with very tight link 
bounds. 

For each problem, we generate a random connected network, where every node has at least one 
path to all other nodes. To that end, all nodes are first connected in a random spanning tree. 
Additional links (creating loops) are then randomly placed. Every link is given a random capacity 
and a random length. Finally a specified number of random requests are generated, in terms of 
start- and endnodes. An example of a generated test problem is shown in fig |^. 

This procedure does not automatically yield a solvable problem, where all requests can be fulfilled 
simultaneously without violating any constraint. In principle, solvability could be built into the 
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Figure 3: An example network with 13 nodes and 17 links. The initial spanning tree are the solid 
lines whereas the dotted lines represent the links creating loops. 

problem-generator. Here we adopt another strategy, by creating random problems and attempting 
to solve them exactly; those with the solution not found by BB within a certain amount of CPU time 
are disregarded. In principle, this method could introduce a bias towards simple problems. However, 
for the problem sizes considered only a tiny fraction of the problem candidates are "timed-out" , 
except for the largest problems (last row in Table |3|), where about 1/3 are disregarded. 

Prior to attacking a problem, a decomposition into independent sub-problems is attempted, to 
reduce complexity. The required computer time for a decomposed problem is dominated by the 
largest sub-problem. All sub-problems are solved in the explorations described below. The CPU 
demand of the MF approach scales like NrNNl. 

The performance of the MF Potts approach is probed my measuring the excess path length as 
compared to the BB result, 

A= Dmf~ Dbb 
-Dbb 

where -Dmf and -Dbb are the total path lengths resulting from the two algorithms. In Table ^ we 
show characteristics of the generated test problems, together with performance measures for the 
Potts MF algorithm. As a measure of the complexity of a problem we use the entropy, S, defined 
as the logarithm of the total number of possible configurations, disregarding load constraints. 



Nodes 


Links 


Requests 


< S > 


% legal 


< A > 


<CPU-time> 


5 


10 


5 


14 


100.0 


0.003 


0.1 


5 


10 


10 


28 


99.9 


0.002 


0.2 


10 


15 


10 


28 


100.0 


0.004 


0.4 


10 


20 


10 


48 


100.0 


0.003 


0.5 


15 


20 


10 


27 


99.8 


0.03 


0.5 


15 


20 


15 


40 


99.9 


0.06 


0.7 



Table 1: Results averaged over 1000 problems of each size. The entries "% legal" and <CPU time> 
refer to the MF Potts approach. The time is given in seconds using DEC Alpha 2000. Typical times 
for the BB method are around 600 seconds. 



Table |l| indicates an excellent performance of the MF Potts approach, with respect to giving rise to 
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legal solutions with good quality, with a very modest computational demand. 



Summary and Outlook 

We have developed a Potts MF neural network algorithm for finding approximate solutions to the 
Multiple Shortest Path Problem. The starting point is a Potts MF recast of the exact Bellman- 
Ford algorithm for the simpler single Shortest Path Problem. This approach is then extended 
to the Multiple Shortest Path Problem by utilizing several Potts networks, one for each request. 
Complications of topological nature are successfully handled by means of a convenient propagator 
approach, which is crucial for the following issues: 

• The MF approach yields at T ^ fuzzy spanning trees, from which the propagator is used to 
extract fuzzy paths. 

• The load-constraints are handled by energy terms involving the propagator. 

• Loops are suppressed by energy terms, also based on the propagator. 

In addition, an auxiliary link to an escape node is introduced for each proper node, opening up 
escape paths for unresolvable situations. 

The method is local in that only information available from neighbouring nodes is required for 
the updates. This attractive feature, inherited from the BF algorithm, facilitates a distributed 
implementation. 

The computational demand of the method is modest. The CPU time scales as NrNNl. With 
fixed connectivity this corresponds to NrN 2 , whereas for the worst case of full connectivity it yields 
N R N 3 . 

The performance of the algorithm is tested on a set of synthetic challenging problems, by comparing 
to exact results from a branch-and-bound method, for various problem sizes. The comparison shows 
that the Potts MF approach with very few exceptions yields very good approximate solutions. 

The method is presently being generalized to other routing problems. 
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Appendix A. The Potts MF Algorithm 



Initialization 



The initial temperature Tq is first set to Tq — 50. If the saturation E, 



1 



(AI) 



E = 



N R (N - I) 



has changed more than 10% after all neurons have been updated once, then the system is reinitialized 
with To — > 2Tq. 

For all nodes (except the end- and escape nodes), the corresponding Potts neurons are initialized 
in accordance with the high temperature limit, i.e. 



for all n, neighbours j (including the escape node) of i . P[j and D\ are initialized consistently with 



Iteration 



Until T < T f or E > S/ do: 

• For every request r do: 

1. For every node i except b r and the escape node: 

(a) Update v[ (eqs. (||J|)). 

(b) Update DV (eq. @). 

(c) Update P 4 r (eq. @). 

2. Update Ly. 

• Decrease the temperature: T = kT. 

Parameters used are: k — 0.90, T/=0.0001, E/=Q. 99999. For the energy coefficients in eq. Jl3|), we 
have consistently used a = 1 and 7 = 5. 



r 



(A2) 



eq. ©. 
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